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Abstract 

We  present  initial  progress  on  so-called  extended  visual 
servoing,  an  integrated  approach  of  controlling  mobile 
robots  equipped  with  multiple  sensors.  Our  approach 
to  sensor  fusion  uses  the  mathematical  framework  of  vi¬ 
sual  servoing  for  a  higher  dimensional  feature  space  that 
incorporates  other  sensors  in  addition  to  vision  (which 
still  plays  a  central  role).  Analysis  and  simulation  on 
a  planar  mobile  robot  with  “vision  +  sonar”  shows  the 
initial  success  of  the  approach. 

1  Introduction 

This  technical  report  builds  on  visual  servoing,  which 
takes  advantage  of  the  rapid  advancement  of  camera  and 
image  processing  technologies  in  order  to  provide  feed¬ 
back  information  for  a  robot  controller  [7,  8].  There  are 
two  approaches  to  visual  servoing:  position-based  and 
image-based.  Position-based  methods  use  a  geometric 
model  of  the  target  to  estimates  the  position  and  ori¬ 
entation  of  the  body  being  servoed,  and  then  compute 
the  control  inputs  in  Cartesian  coordinates  [13].  Image- 
based  visual  servoing  computes  the  control  inputs  di¬ 
rectly  from  the  image  features  [7],  and  it  is  thought  to 
be  more  robust  with  respect  to  camera  calibration  un¬ 
certainty.  Image-based  approaches  generally  require  the 
so-called  image  Jacobian,  the  computation  of  which  usu¬ 
ally  requires  the  depth  of  features  being  tracked  [8] .  Dif¬ 
ferent  methods  address  this  drawback,  such  as  2-1/2D 
visual  servoing  [10],  stereo  [9],  or  the  use  of  a  calibrated 
target  [4,  15].  However,  they  all  introduce  certain  chal¬ 
lenges,  such  as  extra  landmark  points  for  2-1/2D  visual 
servoing,  expensive  hardware  and  extra  computation  for 
stereo  vision,  or  restriction  to  a  limited  selection  of  spe¬ 
cial  landmarks. 

Most  visual  servoing  methods  have  been  developed  for 
robot  arms,  but  may  be  simply  transfered  to  the  control 
of  a  mobile  robot.  In  addition  to  vision,  however,  most 
mobile  robot  platforms  are  equipped  with  extra  motion 
sensors  in  addition  to  vision,  such  as  sonar  [11,  14],  laser 
range  finders,  compasses,  etc.  Sensor  fusion  algorithms 
generally  focus  on  object  pose  estimation  [1,  2],  rather 
than  control.  Therefore,  we  present  a  new  approach  to 
robot  control  by  directly  applying  feedback  from  multi¬ 
ple  sensors. 

Our  method  is  built  on  ideas  of  image-based  visual 
servoing.  The  goal  of  visual  servoing  is  to  move  the 


robot  to  a  position  and  orientation  such  that  the  view  of 
a  scene  obtained  from  a  camera  will  match  a  predefined 
one  that  is  stored  in  memory.  This  report  takes  the 
natural  next  step  to  broaden  our  notion  of  the  “view” 
to  include  sensor  values  from  multiple  sensors,  rather 
than  just  vision.  Due  to  its  versatility,  vision  still  take  a 
central  role,  so  we  call  the  approach  Extended  Visual 
Servoing.  Our  approach  inherits  several  useful  features 
from  image-based  visual  servoing: 

•  Computational  simplicity.  There  is  no  need  to 
transform  sensor  features  to  configuration  space  as 
position-based  visual  servoing  and  most  sensor  fu¬ 
sion  methods. 

•  Independent  of  object  model.  The  precise  geometric 
property  of  the  object  is  not  necessary  since  the 
control  is  based  on  the  sensor  features  only. 

•  Robustness  to  sensor  and  robot  calibration  uncer¬ 
tainty.  The  feedback  loop  is  closed  around  sensor 
readings,  decreasing  sensitivity  to  sensor  and  robot 
calibration. 

By  incorporating  several  sensor  modalities,  it  may 
be  possible  to  take  advantage  of  each  sensor’s  relative 
strengths  to  complement  the  weaknesses  of  others.  Sonar 
applications,  for  example,  are  usually  restricted  to  obsta¬ 
cle  detection  and  avoidance  (rather  than  tracking)  since 
it  is  challenging  to  use  sonar  to  distinguishing  between 
specific  features.  Vision  systems,  by  contrast,  provide  a 
means  by  which  to  distinguish  between  features,  but  are 
poor  feature  depth  estimators.  This  is  a  critical  shortfall 
for  visual  servoing  because  the  image  Jacobian  almost 
invariably  depends  on  the  depths  of  the  features  being 
viewed.  Hence,  we  argue,  sonar  is  an  ideal  complemen¬ 
tary  sensor  to  augment  a  monocular  visual  servoing  sys¬ 
tem.  Thus,  we  exploit  the  synergy  between  vision  and 
sonar,  by  taking  advantage  the  strengths  of  both  sensors 
to  control  a  robot. 

This  paper  is  arranged  as  following.  After  the  intro¬ 
duction  in  this  section,  we  will  model  sensors  and  their 
features  in  Section  2,  then  officially  introduce  the  idea 
of  extended  visual  servoing  in  Section  3.  An  example 
of  applying  the  techniques  with  camera  and  sonar  on  a 
planar  mobile  robot  will  be  given  in  Section  4.  In  last 
section,  we  will  give  some  summaries  and  remarks. 
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2  Feature  Space 

For  a  2D  ground  mobile  robot,  the  perspective  projec¬ 
tion  of  an  object  will  bring  us  one  measurable  variable 
7  in  the  image  plane.  Thus,  the  image  feature  space 
considered  in  this  paper  is  the  simple  projection  of  a 
single  point  feature  onto  a  2D  image  plane,  as  described 
below.  In  addition  to  the  camera,  we  assume  that  a 
sonar  system  augments  the  vision  system  to  provide  an 
extended  feature  space ,  as  described  below. 


2.2  Sonar  Feature 

Though  there  are  many  sensor  suites  from  which  to 
choose,  we  model  our  sonar  system  as  sequentially  trig¬ 
gered,  narrow  beam  transducer  that  can  measure  dis¬ 
tance  from  reflective  surfaces,  as  shown  in  Fig.  2.  The 
sonar  sensors  are  configured  as  an  equally  distributed 
ring  (only  one  sonar,  at  location  (3,  is  explicitly  illus¬ 
trated  in  the  figure).  The  sensitive  area  of  the  sensor  is 
shown  as  an  angle  </>,  which  is  generally  less  than  25°  for 
a  narrow  beam  sonar. 


2.1  Camera  Feature 

For  simplicity,  we  employ  a  simple,  calibrated  2D  pin¬ 
hole  camera  with  perspective  projection  (Fig.  1). 
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Figure  2:  Reflective  model  of  a  narrow  beam  sonar. 
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The  specular  surface  of  the  object  in  field  will  bounce 
the  sound  wave  back  to  the  original  transducer,  to  mea¬ 
sure  the  distance  from  the  sensor  to  the  object, 

l  =  C  •  fTOF/2,  (2) 


Figure  1:  Perspective  projection  for  a  planar  pinhole 
camera. 

As  shown  in  Fig.  1,  the  origin  of  the  camera  frame  is 
given  by  O.  The  X-axis  is  the  optical  axis  that  is  per¬ 
pendicular  to  the  image  plane  and  points  forward  from 
the  lens.1  Let  p  =  [x,y)T  be  the  position  of  a  point  in 
camera  frame.  Then  7  is  its  projection  on  the  image 
plane,  given  by 

7  =  a|.  (i) 

where  A  is  the  focal  length. 


where  c  is  the  speed  of  sound  in  air  and  fxOF  is  the 
“time  of  flight”  of  the  echo,  i.e.  the  time  duration  from 
the  instant  the  sonar  is  activated  to  the  moment  the 
echo  is  received.  If  the  radius  of  the  sonar  ring  is  r,  the 
distance  from  an  object  located  at  (x,  y)  with  respect  to 
the  sonar  center  O  can  be  approximated  as 

8  =  l  +  r  =  (x2  +  y2)  ’  (3) 

as  long  as  l  >  r.  The  angle  of  the  target  is  often  approx¬ 
imated  as  the  angle  of  the  sonar  /?,  but  the  error  is  too 
large  to  be  feasible  in  this  paper. 

2.3  Combining  Vision  and  Sonar 


1  The  choice  of  axes  labels  is  slightly  different  from  the  common 

choice  in  the  literature. 


We  define  the  overall  sensor  information  that  can  be  ac¬ 
quired  for  a  robot  regarding  its  motion  as  the  senso- 
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rium ,  S.  In  the  simple  example  presented  in  this  sec¬ 
tion,  5  =  lx  R+,  since  the  image  plane  is  represented 
by  7  £  R  and  the  sonar  measurement  is  represented  by 
5  >  0.  Further,  we  categorize  all  the  corresponding  prop¬ 
erties  of  the  environment  being  measured  by  sensors  as 
landmarks ,  C.  Again,  in  the  simple  example  in  this 
section,  there  is  a  single  landmark,  which  is  a  point  in 
the  planar  world,  so  C  =  K2.  Therefore,  we  model  the 
sensor  suite  as  a  mapping  from  the  landmarks  to  senso- 
rium,  h  :  C  — >  S.  In  our  case,  assuming  the  camera  and 
sonar  frames  are  collocated  for  convenience,  we  have 


h(p ) 


A  x/y 
Vx  2 +  y2 


where 


(4) 


0  =  se(2),  we  have  u>  €  R,  v 
[0  -u>  vx~\ 


Z  =  g-l9  = 


=  [vx,vy]T  €  M2,  and 
G  g ,  and  (w,v)  G  R(S)]R2  . 


3.2  Landmark  Kinematics 

Consider  the  group  action  of  an  element  g  €  G  that 
translates  a  point  pb  =  [xb,  yb,  Zb,  1]T  G  E3  in  the  robot 
based  coordinate  system  to  the  fixed  world  frame  pw  = 
[xw,yw,zw,  1]T,  via 


Pw  =  gpb- 


(5) 


3  Extended  Visual  Servoing  Since  the  velocity  of  any  fixed  point  in  the  world  frame 

is  zero,  then  its  velocity  in  robot  body  based  frame  can 
In  this  section,  we  present  a  general  approach  to  ex-  be  obtained  by  differentiating  (5),  i.e., 
tended  visual  servoing  that  we  will  restrict  later  to  the 

case  of  a  planar,  unicycle-type  mobile  robot.  pb  =  —g~lgPb  =  ~£,Pb  ■  (6) 


3.1  Rigid  Kinematics 


We  consider  the  problem  of  positioning  a  robot  using 
feedback  from  onboard  sensors.  The  configuration  space 
of  a  mobile  robot  is  modeled  as  Q  =  SE(3)  for  a  spa- 
cial  robot  or  G  =  SE(2)  for  a  planar  robot.  Therefore, 
the  pose  of  the  robot  is  given  by  a  rigid  transformation 
matrix 


g  = 


R 

0T 


d 

1 


&G 


where  d  and  R  are  the  position  and  orientation  of  the 
robot  expressed  in  the  fixed  world  frame,  respectively. 
Recall  that  TQ  ~  G  x  g  ~  G  x  (K"(™_1)/2(s)]R"),  where 
0  =  TjG  is  the  tangent  space  at  the  identity  [12].  We 
make  the  identification  via  the  “left  inverse”,  namely  if 
(g,g)  G  TQ,  then,  for  Q  =  SE(3),  we  have 


Consider  the  trivial  “landmark  function”  £  :  G  — *  £, 
where  L  =  E”  represents  a  single  feature  point  and  G  = 
SE(n),  n  =  2  or  n  =  3.  Then 

Z{g)  =g~1Pw  =Pb-  (7) 

By  using  the  above  tangent  space  identification,  the  tan¬ 
gent  map  T£  :  TQ  — »  TC  admits  a  convenient  matrix 
representation,  Ji,  given  by 


Pb 


where 


Je. 

_ A _ 

Pb 

-73x3] 

■N 

0T 

0T 

. 

'  0 

- 

Zb 

Pb 

= 

Zb 

0 

- Vb 

xb 

Vb 

—Xb 

0 


(8) 


£,  =  g~1g  = 


0  -coz 


IXy 


U)z  0  -U!x 

-LOy  LOx  0 

0  0  0 

and  (u,v)  G  P3/^™3 


vz 

0 


G  0 


Again,  for  SE(2)  we  have  uj  G  K,  v  =  [vx,vv]T  G  M2, 
Pb  =  [xb,Vb,  1  }T,Pw  =  [xw,yw,  1]T  G  E2,  and 


Pb  —  hx2 

0  0T  ’ 


where  pb 


- Vb 

xb 


(9) 


Physically,  £  represents  the  linear  v  =  [vx,vy,vz]T  and  It  is  straight  forward  to  generalize  the  above  expres- 
angular  w  =  [u>x,  u>y,  uiz]T  velocities  relative  to  the  in-  sions  for  landmark  functions  l  that  incorporate  multiple 
stantaneous  body-frame.  Similarly,  for  G  =  SE(2)  and  landmark  features. 
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3.3  Feature  Kinematics 

Suppose  C  C  E"  is  some  set  of  landmarks,  that  is,  points 
in  the  world,  fixed  in  a  world  frame,  and  S  is  the  sensor 
feature  space.  Let  t  :  G  — >  C  denote  the  mapping  from 
robot  configuration  to  relative  landmark  configuration, 
as  described  above.  Let  h  :  C  — »  S  be  the  sensorial 
mapping.  Then  Th  :  TC  — >  TS  is  the  tangent  map  (or 
Jacobian)  relating  landmark  velocities  to  feature  veloci¬ 
ties,  with  matrix  representation  Jh ,  namely 


Sec.  4  has  fewer  control  degrees  of  freedom  than  rigid 
body  degrees  of  freedom.  Let  M  be  the  mapping  be¬ 
tween  control  input  and  body  velocity.  Then  the  follow¬ 
ing  commutative  diagram  describes  the  flow  of  velocities 


U- 


M 


TQ 


Tt 


TC 


Th 


TS 


G 


h 


s 


s  =  Jhp. 


(10)  or,  in  coordinates 


where  s  =  h(p)  denotes  the  sensor  measurement. 

The  Jacobian  for  our  simple  “sonar  +  vision”  senso- 
rium  defined  in  the  previous  section  can  be  found  by 
differentiating  (4),  namely 


s  = 


l(y2 


'  X2)  2  (yy 


+  xx) 


or 


where 


8  =  JhP 


A  = 


2  _22  rY 
5  A<5  u 


0 


is  the  sensor  Jacobian  obtained  by  noticing 


r  1 

X 

r— 1 

n 

p  = 

y 

11 

7 

11 

07 

V 

1 

1 

.  1  _ 

(11) 


(12) 


J 

s  =  JhJeM  u . 

There  are  now  many  possible  control  inputs.  Suppose 
that  s*  is  a  desired  velocity  in  sensor  space.  Then,  we 
could  let 

u  =  JH*  (13) 

where  jl  is  the  pseudo-inverses  of  J.  In  some  cases,  it 
may  actually  be  possible  to  find  an  exact  inverse  to  J . 
For  example,  if  the  spaces  G  ~  C  ~  S  are  diffeomorphic, 
and  l  and  h  are  diffeomorphisms,  and  the  system  is  fully 
actuated,  so  that  M  is  nonsingular,  then  we  can  compute 
J-1,  as  we  have  done  for  visual  servoing  in  several  prior 
contexts  contexts  [4,  5,  15].  In  this  report,  we  consider 
a  case  in  which  J  is  full  rank,  even  though  G  and  C  are 
not  diffeomorphic. 


where  rj  =  (y2  +  A2)  2 . 


4  Example:  Rolling  Penny  Robot 


3.4  Extended  Visual  Servoing 

The  control  objective  is  to  move  a  robot  to  align  the 
sensor  measurements,  s  =  h  o  £(g),  with  a  predefined 
value  s*.  If  we  define  U  as  the  control  input  space  of  the 
robot  and  Q  its  configuration  state  space,  then  the  con¬ 
trol  input  u  €  IA  of  a  mobile  robot  will  change  the  pose 
(position  and  orientation)  g  €  G  of  the  robot  through 
robot  kinematics  and  dynamics,  which  in  turn  will  up¬ 
date  the  location  of  landmarks  C  with  respect  to  the 
robot  frame.  Then  the  on  board  sensors  will  further 
map  the  landmarks  to  sensor  space  s  £  S. 

For  the  present  work,  we  posit  a  purely  kinematic 
plant  model,  actuated  in  the  body  frame.  For  fully  ac¬ 
tuated  systems,  A\mU  —  dimg,  but  we  do  not  limit  our¬ 
selves  to  that  case.  In  fact,  the  example  investigated  in 


Consider  a  cart-like  “rolling  penny”  or  “Hilare”  robot. 
The  configuration  of  the  robot  and  its  equivalent  rolling 
penny  illustration  are  shown  in  Fig.  3.  A  camera  is  fixed 
to  the  robot  body  such  that  its  coordinate  system  is  iden¬ 
tical  to  the  body  frame,  and  its  optical  axis  is  aligned 
with  the  X-axis  of  the  robot.  In  addition,  there  is  a  sonar 
ring  with  origin  also  coincident  with  the  body  frame  ori¬ 
gin. 

For  illustration  purposes,  we  assume  that  there  is  only 
one  object  in  the  field  of  view  of  the  robot,  and  that 
the  camera  observes  a  single  feature  of  the  object;  for 
example  one  could  use  the  image-plane  center  of  mass 
as  done  in  [4,  15],  to  obtain  the  overall  direction  of  the 
center  of  mass  of  the  object  relative  to  the  robot.  We 
consider  convex  objects,  toward  the  robot,  such  that  the 
reflecting  point  of  the  object  is  coincident  with  the  center 
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Figure  3:  Top  view  of  the  configuration  of  a  cart-like  robot  (left).  The  equivalent  rolling  penny  model  (right). 


of  the  color  mass  as  illustrated  in  Fig.  4.  However,  we 
believe  this  restriction  may  be  relaxed  or  modified  to 
accommodate  a  different  selections  of  image  features. 

For  simplicity,  as  mentioned,  the  sonar  and  camera  are 
configured  in  a  way  such  that  the  center  of  the  sonar  ring 
is  the  origin  of  the  camera  frame.  Thus,  the  coordinate 
systems  for  the  camera,  sonar  and  robot  are  identical. 
Further,  although  the  sonar  sensors  are  always  installed 
as  a  ring  and  theoretically  could  obtain  a  180°  or  360° 
field  of  view,  we  restrict  ourselves  to  the  portion  that 
coincident  with  the  field  of  view  of  the  camera. 


and  the  angular  velocity  U2,  thus 


M 


_ 

"0 

1' 

LU 

— 

1 

0 

V 

0 

0 

Recall  that  s  =  JhJeMu  where  Jh  is  given  by  (11) 
and  Je  is  given  by  (8).  Thus  we  have 

s  =  Ju  (14) 


where 

Recall  the  mappings  s  and  (.  for  our  “camera  +  sonar” 
sensorium,  measuring  a  single  point,  are  given  by  (4)  and 
(7),  summarized  for  convenience  here: 


J  =  Jh  Ji  M  = 


T1 

S_\ 

V 


Note  that  dim  Cl  >  dim  S.  However,  we  have  chosen 
dim S  =  dim U  =  2.  Thus,  noting  that  y2  =  A2  +  72  >  0, 


X 

the  inverse 

p  = 

y 

1 

=  t(g)  ■=  9  1Pw, 

J-1  = 

0  "*|b- 
_ 1 

T)~ 

A 

717 

(5 

=  h(p)  := 


|A 

( x 2  +  y 2)5 


is  well  defined  for  <5  >  0.  Given  desired  features  s*  = 
[7*,  (5*]T,  we  can  apply  a  simple  proportional  control 
law  with  positive  definite  gain  matrix  K ,  namely 


where  g  G  Q  =  SE(2),  p  G  C  =  E2  and  s  G  5  =  R  x  R+.  u  =  J  1I<{s*  -  s )  (15) 

Thus,  s  =  h  o  t(g). 

which  implies 

s  =  — K's 


The  two  inputs  of  the  robot  are  the  forward  speed  u\ 


(16) 
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Figure  4:  Top  view  of  a  planar  mobile  robot  with  both 
camera  and  sonar. 


put  translational  and  rotational  velocities  of  the  robot 
with  time.  The  subplot  “Image  Feature”  is  scaled  to 
pixel  coordinates,  assuming  a  true  focal  length  of  5mm, 
and  120pixels/mm.  The  “Distance”  subplot  illustrates 
the  change  of  7,  the  horizontal  position  of  the  object  in 
the  image  plane  (corresponding  to  the  sonar  measure¬ 
ment)  . 

In  Fig.  6,  the  subplot  “Body”  shows  the  trajectory 
of  the  target  observed  by  the  robot  in  the  robot  body 
frame.  The  subplot  “Global”  shows  the  trajectory  of 
the  mobile  robot  in  the  global  frame  during  the  process 
of  moving  and  turning  to  approach  the  target  with  the 
desired  distance  and  orientation. 

As  shown  by  this  anecdotal  simulation,  the  system 
converges  as  expected.  Notice  that  since  we  are  only 
controlling  2D  OF,  there  is  a  one  dimensional  symme¬ 
try  that  we  do  not  control  -  it  corresponds  to  a  circle 
around  the  target  for  which  the  robot  is  facing  inward. 
If  the  target  were  moving  in  time,  the  algorithm  could 
be  modified  for  tracking,  as  done  in  [3]. 


where  J  =  (s  —  s*)  is  the  sensorial  error. 

To  avoid  collisions  with  the  feature  object,  and  to  keep 
the  feature  within  the  field-of-view,  one  could  easily  in¬ 
troduce  a  Navigation  Function,  as  done  in  [5].  In  this 
case  it  is  straight  forward  since  the  space  5  =  f  x  R+  is 
so  simple. 

5  Simulation  Experiments 

For  our  simulation  experiments,  a  target  resides  at  pw  = 
[6,3]Tm  in  the  world  frame.  The  initial  location  of  the 
robot  body  frame  was  coincident  with  the  world  frame 
at  t  =  0.  As  described,  we  assume  for  convenience  that 
the  coordinate  frames  of  the  sonar  ring  and  camera  coin¬ 
cide  with  the  body  frame.  Since  the  camera  is  assumed 
calibrated,  we  scale  the  image  measurement  to  achieve 
an  effective  focal  length  of  A  =  lm. 

Let  s*  =  [0,  2]Tm,  a  goal  that  corresponds  to  moving 
the  mobile  such  that  the  object  is  right  in  front  of  the 
robot,  7*  =  0,  at  a  distance  of  S*  =  2m.  Then  applying 
the  approach  above,  we  can  easily  obtain  closed  loop 
control  for  the  task  with  a  given  tolerance  (that  will 
depend  in  practice  on  the  true  pixel  resolution  of  the 
camera,  and  the  sonar  resolution). 

In  Fig.  5,  the  subplot  “Control  Inputs”  shows  the  in- 


6  Summary 

This  work  builds  on  and  enhances  many  techniques  de¬ 
veloped  for  visual  servoing.  For  example,  one  can  still 
apply  Navigation  Function  to  avoid  obstacles  and  keep 
features  in  the  field-of-view  [6] ,  or  integrate  motion  plan¬ 
ning  to  achieve  long  distance  movement  and  obstacle 
avoidance  [16].  By  applying  extended  visual  servoing, 
we  can  take  advantage  of  mathematical  framework  of 
visual  servoing,  without  limiting  ourselves  to  a  single 
sensor  modality.  There  are  several  advantages  to  this 
approach. 

First,  using  sensor  features  from  multiple  sensors,  we 
can  obtain  a  Jacobian  with  increased  rank,  hence  map 
the  feature  velocity  to  a  larger  subset  of  the  robot’s  mo¬ 
tion.  With  a  bit  of  ingenuity,  it  may  in  some  cases  be 
possible  to  find  a  diffeomorphism  between  the  robot’s 
motion  and  the  sensorial  feature  motion.  In  the  case  of 
an  underactuated  robot,  like  the  example  we  studied,  we 
described  an  extended  feature  constellation  that  allowed 
us  to  use  feedback  linearization  on  a  subset  of  the  avail¬ 
able  configurations,  giving  rise  to  a  one-dimensional  set 
of  solutions. 

A  second  advantage  arises  by  selecting  sensors  that 
complement  each  other.  Even  in  cases  where  there  is 
a  known  diffeomorphism  between  sensor  and  robot  co- 
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Figure  5:  Control  inputs  and  controlled  sensor  features. 
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Figure  6:  Target  trajectory  in  body  frame  and  robot  trajectory  in  global  frame. 
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ordinates,  representing  the  Jacobian  may  be  “messy” 
in  image  coordinates  [5],  or  may  require  special  targets 
[4,  16].  However,  the  sensor  Jacobian  in  the  present  pa¬ 
per  may  be  compactly  written  making  very  few  assump¬ 
tions  about  target  geometry. 

Third,  as  we  noticed  in,  Section  2  and  3,  although 
the  landmarks  can  be  expressed  in  a  global  frame,  the 
expressions  are  not  used  in  derivation  of  extended  vi¬ 
sual  servoing.  Hence,  the  algorithm  is  insensitive  to  the 
geometric  model  of  the  landmarks.  Admittedly  the  land¬ 
mark  feature  set  was  trivial  in  the  present  context,  and 
so  more  must  be  done  to  see  how  these  ideas  can  be 
generalized. 

Fourth,  by  integrating  the  sensor  features  in  the  body 
based  coordinate  system,  we  can  bypass  the  common 
difficulties  of  sensor  fusion,  e.g.  the  extra  step  of  trans¬ 
ferring  the  feature  information  back  to  the  global  frame, 
and  then  calculating  the  control  input  in  the  global  coor¬ 
dinate  system,  which  has  to  be  remapped  into  the  robot 
frame. 

Finally,  although  we  derived  the  extended  visual  ser¬ 
voing  control  algorithm  based  on  a  kinematically  actu¬ 
ated  planar  mobile  robot,  we  believe  that  extending  it  to 
both  dynamically  actuated  mobile  robots  such  as  a  sub¬ 
marine  or  a  blimp,  or  employing  the  algorithms  for  a  the 
fixed-camera  and  moving  robot  arm  should  be  possible. 
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